Nature of the antiferromagnetic quantum phase transition on the honeycomb lattice 
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We address the nature of the antiferromagnetic quantum phase transition that separates a 
semimetal from an antiferromagnet in the repulsive Hubbard model defined on the honeycomb 
lattice. At the critical point, the fermions acquire an anomalous dimension rj due to their strong 
coupling to the fluctuations of the order parameter (j>. The finite r] in turn induces a singular i^* 
term and a non-analytical spin susceptibility signaling the breakdown of Hertz's theory. As a 
result, the continuous antiferromagnetic quantum phase transition is internally unstable and turns 
into a first order transition. 

PACS numbers: 71.10.Fd, 71.10.Hf, 73.43.Nq 



Quantum phase transitions are believed to be present 
in a wide variety of correlated quantum matter rang- 
ing from strongly correlated electron systems to ultracold 
quantum gases on engineered lattices [l|. Such quantum 
phase transitions occur at zero temperature as a function 
of a non-thermal tuning parameter [ll-U • Understanding 
the nature of possible quantum phase transitions of a 
given system and describing the ensuing physical behav- 
iors present a great challenge of modern materials science. 
In particular, the possible zero-temperature phase tran- 
sitions in various models of interacting electrons on the 
two-dimensional honeycomb lattice have recently gener- 
ated intense theoretical and numerical interests |4l-[l4|. 
While it is well-established that the ground state of the 
repulsive Hubbard model is a semimetal (SM) at weak 
coupling and a Mott-insulating antiferromagnet (AFM) 
at strong coupling, the nature of the transition between 
these two phases is still not well understood. 

There are generally two scenarios for the transition 
from a SM to an AFM. Herbut studied the repulsive 
Hubbard model on a honeycomb lattice by means of a 
renormalization group (RG) method in the large- A'^ limit 
and argued that the strong on-site interaction drives a 
direct, continuous SM-AFM quantum phase transition, 
which is in qualitative agreement with the results ob- 
tained by a functional RG calculation iS] and early quan- 
tum Monte Carlo (MC) simulations [9|- Treatments of 
the Hubbard models based on auxiliary-particle meth- 
ods on the contrary seem to indicate that various exotic 
quantum spin liqu id p hases may exist between the SM 



and AFM phases [1 
tum MC simulation 



13| . Particularly, a recent quan- 
14| found an intermediate gapped 
spin liquid phase. These two scenarios are apparently 
contradictory, and no consensus has been reached so far. 

The theoretical study of q uantum phase transitions 
was pioneered by Hertz [l5|, who developed a field- 
theoretic approach to describe the (anti-)ferromagnetic 
phase transition in an itinerant electron system. In this 
approach, the fermionic degrees of freedom are com- 
pletely integrated out, yielding an effective (j)'^ action so- 
ley in terms of an order parameter (ft. Hertz's theory and 
its generalizations |17| have been applied to numer- 



ous quantum phase transitions Recently, however, 

the general validity of this approach has been called into 
question [1, IT8| - |20| as it has become clear that it is not 
always eligible to integrate out all fermionic degrees of 
freedom in an itinerant electron system. Indeed, the sin- 
gular interaction between gapless itinerant fermions and 
order parameter fluctuations can lead to important fea- 
tures that are beyond a simple (j)'^ description [T8l - [20| . 

In this Letter we address the nature of the quan- 
tum phase transition between an antiferromagnet and a 
semimetal on the honeycomb lattice. We approach the 
problem by assuming that the SM-AFM phase transi- 
tion is a continuous one and then examining whether it 
is stable or not. To this end, we maintain the fermionic 
degrees of freedom of the system, described as massless 
Dirac fermions, within a Yukawa-type field theory to de- 
scribe the SM-AFM quantum critical point. This allows 
for a careful study of the singular interaction between 
Dirac fermions and order parameter fluctuations. 

We first compute the fermion anomalous dimension 
after incorporating the damping effects of both Dirac 
fermions and order parameter. We then calculate the co- 
efficient of an effective quartic cj)'^ term and the spin sus- 
ceptibility, and show that both of these two quantities are 
non-analytical. As it turns out, the non-analytical spin 
susceptibility leads to an effective energy of 4> that has a 
minimum at a finite momentum qc, so a nonzero AFM or- 
der parameter is generated at the transition point. This 
order parameter is nonuniform and spatially modulated 
as ((/>) = (poe^"'^. All these unusual behaviors indicate 
that, the original Hertz cj)'^ theory breaks down and the 
Dirac fermions and order parameter fluctuations should 
be treated equally. As we demonstrate below, the puta- 
tive continuous SM-AFM quantum phase transition is de- 
stroyed by the critical order parameter fluctuations and 
consequently becomes first order. 

We begin with the half-filled Hubbard model 

^ = - E + E ^ ("»t - 1) - 1) (1) 

defined on a two-dimensional honeycomb lattice. Here, 
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n-ia = cj^Cia and cj^ {ci,cr) creates (destroys) an electron 
of spin projection a at site i. The sum < i,j > is over 
nearest neighbors and the hoping t is taken to be a con- 
stant. A band structure analysis shows that the conduc- 
tion band touches the valence band at discrete points, so 
the low-energy excitations are massless Dirac fermions. 
Around the touching Dirac points, the kinetic term can 
be converted to a free Lagrangian term 

-Co = ^ ^cridrlO - iVpdxll - iVFdy-l2)'^a, (2) 
a 

where 70 = 1 18) T3, 71 = 1 (X) iri, and 72 = T3 (g) 1x2- 
The 4-component spinor represents Dirac fermion, 

and is arranged as = {cAKa,CBKcr,CAK'a,CBK'crY 
with sublattice indices B and valley indices K, K' . Its 
conjugate is = ^3t7o- The on-site Hubbard interac- 
tion can be rewritten as the sum of a number of contin- 
uum four-fermion interaction terms, among which only 
the following term turns out to be relevant in the RG 
sense [B], 



(3) 



where g = Ua'^/8 and a is the lattice constant. 

We first sketch Hertz's approach [l^. It is convenient 
to introduce a scalar field </> and perform a Hubbard- 
Stratonovich transformation, yielding 



C = Co 



N 



Integrating out the fermion fields and taking the 

saddle point approximation leads to a gap equation [2l| 



g 



16 



(0) 



(5) 



The critical point separating zero and finite expectation 



value (0) is given by — = 16 



For g < 



the system is in the gapless SM phase with (0) = 0. For 
g > gc, {4>) 7^ 0, so the Dirac fermions acquire a finite 
mass gap and the system is turned to a Mott insulator. 
Such excitonic-type insulating transition is a condensed- 
matter analog of the dynamical fermion-mass generation 
mechanism proposed originally in particle physics [2^ . 
The order parameter can be expressed in terms of fermion 
operators as (0) = (X^o- ''"^c^^^), so the Mott insulator 
is indeed an AFM. Close to g^ the fermion gap is ap- 
proximated by ((/)) = ^ — , which suggests that 
the SM-AFM transition is continuous. This phase tran- 
sition is tuned by the parameter with g^ 

defining the zero-temperature quantum critical point. 

We will now demonstrate that Hertz' scenario of the 
SM-AFM transition on the honeycomb lattice can be 
fundamentally altered by the coupling between Dirac 
fermions and critical order parameter fluctuations. This 
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FIG. 1: (a) the dressed boson propagator; (b) the one-loop 
diagram for the polarization function; (c) the fermion self- 
energy diagram. 



coupling describes the decay of composite order param- 
eter into single fermionic excitations as well as the for- 
mation of order parameter by fermion pairing. Since it 
affects the dynamics of both fermions and bosons, we are 
led to maintain the fermions in the Yukawa-type theory, 
Eq. (4), and study the fermions and bosons on equal foot- 
ings. This will be done by analyzing the Yukawa interac- 
tion by means of an 1 /N expansion where N represents 
the fermion flavor (spin). 

Due to the Lorentz invariance of the action in the low- 
energy sector, the dynamical exponent is 2: = 1. The free 
fermion propagator is C?o(w,k) = In leading 

order of 1 /N expansion, the polarization is given by 



no(r!,q) 



Tr[Go(c^,k)Go(c^ + f7,k + q)], (6) 



and shown in Fig. 1(b). It follows that Ho(r2,q) = 
j\/ + . The bosonic propagator at the same order 
in 1 /N is therefore D"! (f7, q) = iV [n"^ + + n^{VL, q)] . 
In the low energy regime, the free terms turn out to 
be unimportant, so D(il,a) = , The lead- 

ing fermion self-energy diagram is shown in Fig. 1(c). 
Retaining the most divergent part leads to I]o(w,k) = 

~3Av(^'^To-7 • k) In {^-^^—^ . From the Dyson equa- 
tion, G~^{ijj, k) = iLu^Q — 7 • k — k), this results in 
a dressed fermion propagator 



G(w,k) = - 



iuj^Q — 7 • k 

(w2+k2)l-^ 



(7) 



with anomalous dimension 77 = ^^^^ . This propagator 
implies the absence of a well-defined quasiparticle pole. 

So far, our calculation is on the level of random phase 
approximation since the polarization Ho(0,q) has been 
obtained from the free propagator Go(w,k). However, 
Ho(ri,q) may be considerably modified when the feed- 
back of fermion dressing is included. Replacing Go(w,k) 
by G(a;, k) shown in Eq.(I7]), we obtain a new polarization 



H;,(l],q)=A(,7)(02+q2)^ 



(8) 



where A{r]) is a well-defined function of 77 ^25|. The 
fermion self-energy and anomalous dimension 77 may 
also be altered, so it is necessary to recalculate the 
fermion self-energy by including the dressing effects of 
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both fermions and bosons. Using the dressed expres- 
sions, Eqs. (7) and (8), we get a new fermion self-energy 
I](a;,k) = -^i^ii£l£iz:2:k another 

function of rj [1^. Replacing Sol'^jk) by I](a;,k), we are 
again lead to Eq.Q- The dressed fermion propagator 
G(aj, k) thus survives the self-consistency analysis. 

We are now in a position to examine the validity of 
Hertz's 0'* theory and the stability of the continuous SM- 
AFM quantum phase transition. According to Hertz [l^ , 
one can formally integrate out all fermionic degrees of 
freedom and obtain an effective action as a function of 
the order parameter (p: 



5, 



effm = 



0,q 



[X2{n, q)</)2 + X4(f^, q)^" + 0(</>')] . (9) 



Here, the quadratic coefficient is determined by the spin 
susceptibility: X2(f^, q) = r2^-|-q^-l-H(r2, q). The validity 
of this (j)^ theory is rooted on the basic assumption that 
the coefficients X2(^, ^) and X4(^i q) are regular in f2, q. 

We first consider the coefficient of the quartic term. 
According to the leading diagram shown in Fig. 2, it is 

X4(f^,q) = y'^^Tr[G(c.,k)G(^ + 0,k + q) 

xG(w,k)G(a; + l^,k + q)]. (10) 

Using the dressed fermion propagator Eq.Q and carry- 
ing out some tedious calculations, one can show that 



(fi2 +q2)i 



with 



2r(4- 2?7) 

i3r(|- 



3 r(|~277) 
2 r(3 - 2t]) 

2vr 



-2n' 



r(| 



(11) 



2v) 



2 r(4 - 277) 

r(i-2?7) 



r(4 - 2,;) 
dx 



[x{l — x)] 2 >) 
r(i-2r;)n '■I 



r2(2-r;) r2(l-?7) 



dx 



[x{l-x)Y- 



The function F{rii) is finite for any finite 77 and it is easy 
to see that X4(^i q) is divergent in the il, q — >■ limit. In 
other words, the quartic term is singular. An impor- 
tant feature of F{ri) is that it vanishes in the absence of 
fermion anomalous dimension, i.e., F{rj = 0) = 0, thus, 
this singularity is absent if the free propagator Go(a;,k) 
is used. Therefore, the singularity in X4(^iQ) arises 
directly from the critical order parameter fluctuations, 
which is reflected in the finiteness of rj. 

We now turn to analyze the spin susceptibility at the 
quantum critical point. For the effective 0^ theory to be 
applicable and the continuous SM-AFM transition to be 
stable, the static spin susceptibility X2{^) should have a 
regular momentum dependence. It is commonly believed 
that the spin susceptibility of a Fermi liquid is regular. 




FIG. 2: (f)*' vertex correction at the one-loop level. 



FIG. 3: Polarization functions up to the two-loop level. 



Interestingly, Belitz et al. demonstrated that a nonan- 
alytic spin susceptibility in a clean Fermi liquid system 
can occur [l^ . Subsequently, Chubukov et al. obtained a 
negative, nonanalytic spin susceptibility for a ferromag- 
netic quantum phase transition system with dynamical 
exponent 2 = 3 [l^. It is therefore necessary to check 
whether the spin susceptibility is regular or singular. 

The spin susceptibility actually corresponds to the po- 
larization function H(q). The leading diagram is shown 
as Fig. 3(a), with Hj (0, q) being already given by Eq.(8). 
The two-loop contributions are represented by the three 
diagrams (b), (c), and (d) of Fig. 3. Using Eqs. (7) and 
(8), we find that the sum of (b) and (c) is 



H^+^(r!,q) =-2Gi(r7)H^(r!,q) 



(12) 



where Ci{r]) — g;;^^^^^. Employing a method used in 
[23} and then performing a lengthy computation, we find 



n^(r!,q) = -2G2(?7) In 



A 



H^(17,q), (13) 



where C2{i]) is a function of 77. Adding H^, Hj''"^, and Hg 
together one arrives at the expression 



H2(r!,q) = (r!2 + q2 



n^(17,q)-2Gi(77)H5(f^,q) 

^-2Gi(r7)] (^^2 + q2)^+^ 

(14) 



where ry' = —2C2{i]). Since 77' < 0, the function H2(57, q) 
is positive for large fi, q but becomes negative for r2,q 
below a given threshold. As a result, the corresponding 
spin susceptibility X2(f^, q) is non-analytical. 

As with the X4 term, this non-analyticity is directly re- 
lated to the fermion anomalous dimension: If we assume 
77 = and apply the free fermion propagator Go(w,k), 
then H|(51, q) is identical to H"(ri, q) given by Eq.(6), 
and both H2^'^(il,q) and H2(fi,q) are proportional to 

In (^-^^=jll^{n,q). The total polarization for 77 = 

wiU be H2(f2, q, ?7 = 0) oc (fi^ -|- q^)2 + 3^^^ which is 
positive and analytical. 
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Based on the above analysis, we arrive at a key con- 
clusion: the non-analyticities in Xii^il) and X2{^j'i) 
clearly signal the breakdown of Hertz's (j)'^ theory. More- 
over, the non-analytical spin susceptibility will alter 
the nature of the SM-AFM transition. To demon- 
strate this, we write the effective action for (p as 
5effM = J^[{q'+n2{q))(f>^+X4{q)<l>^] with U^iq) 
given by Eq.(14). To determine its ground state, one 
needs to find the minimum of the total energy. The ab- 
solute minimum of kinetic energy Eg = q^ + (g) is 
located at a finite momentum qc, which is one of the so- 
lutions of equation dEq/dq = 0. Though it is hard to 
derive an analytical expression of (jc, its finiteness can 
be easily confirmed by numerical computation. Insert- 
ing qc to Eq, we find that the non- analytical spin sus- 
ceptibility leads to a negative minimal kinetic energy, 
i.e., -E™™ < 0. Around Qc, Eq can be approximated by 
Eq « -I- K,{q — qcY with certain constant k. Origi- 

nally, at the critical point g = gc, the scalar field has a 
vanishing mass, r = 0, so the potential energy is simply 
V[(j>] cx (j)'^ and one should expect (</>) = 0. However, the 
finite E"^™ induced by non-analytical spin susceptibility 
serves as an effective, negative mass. Moreover, Xiil) is 
regular for finite q^ and hence can be safely replaced by a 
constant. Consequently, the effective potential becomes 
V[(t)] = - 02 + X4('7c which yields a finite ex- 

pectation value, (0) = [— i?™'"/2X4(9c)] ^ , even if r = 0. 
According to the generic analysis of Brazovskii [23| , this 
nonzero order parameter (0) should be nonuniform. By 
minimizing the corresponding free energy, we find the or- 
der parameter is spatially modulated as 0(x) — (j)Qe^'^'''^ . 

In the SM phase with r > 0, the order parameter 
fiuctuation is gapped, so the Yukawa interaction does 



not generate any fermion anomalous dimension. As ex- 
plained in the above discussions, X2(w,q) and X4(a;,q) 
exhibit no singular behaviors when r] = 0, thus the spin 
susceptibility is analytical for r > and non-analytical 
only at the critical point r ~ 0. Therefore, (0) van- 
ishes for any positive r, but develops a finite magnitude 
at r = discontinuously because of the negativeness of 
-E""". Apparently, although SM-AFM phase transition is 
shown to be continuous by the gap equation analysis, it 
is ultimately driven first order due to singular interaction 
between order parameter and massless Dirac fermions. 

In summary, we analyzed the nature of the quantum 
phase transition between a semimetal and an antiferro- 
magnet on a honeycomb lattice. We demonstrated that 
the strong interaction between massless Dirac fermions 
and critical order parameter fluctuations generates unex- 
pected properties. As a consequence, the original Hertz 
(j)'^ theory is incomplete for this problem and the mass- 
less fermions should be treated on equal footing with 
the critically fluctuating order parameter. This behavior 
arises from the non- vanishing fermion anomalous dimen- 
sion, which in turn reflects strong damping effects that 
need to be included. Our results indicate that, although 
the SM-AFM transition is continuous at the mean-fleld 
level, it is destroyed by the critical order parameter fluc- 
tuations and actually gives way to a flrst order transi- 
tion with a spatially modulated order parameter. The 
anticipated quantum critical phenomena of continuous 
SM-AFM transition 0-01 are not expected to exist. 
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